Scene change detection (also referred to in the literature as “shot change detection”) is the process of identifying changes in the scene content of a video sequence. Scene change detection is primarily used to segment video into scenes, and facilitate browsing of video sequences and retrieval of desired video content.
Scene changes can occur as abrupt changes when a frame from one shot is followed by a frame from a different shot, and as gradual transitions such as fade-ins and fade-outs.
A survey of scene change algorithms is provided in Gargi, U., Kasturi, R. and Strayer, Susan, “Performance Characterization of Video-Shot-Change Detection Methods,” IEEE Transactions on Circuits and Systems for Video Technology, Vol. 10, No. 1, February 2000, pages 1–13. One class of algorithms used to perform automatic scene detection is color histogram methods. The difference between color histograms of consecutive frames is determined, and large frame differences are flagged as possible scene changes. Another class of algorithms performs scene detection directly on MPEG compressed video, using discrete cosine transform (DCT) and motion prediction information.
A reference for various metrics used for scene change detection, is Ford, R. M., Robson, C., Temple, D. and Gerlach, M., “Metrics for Scene Change Detection in Digital Video Sequences,” in Proceedings of IEEE International Conference on Multimedia Computing and Systems, Jul. 3–6, 1997, Ottawa, Ontario, Canada.
U.S. Pat. No. 5,956,026 of Ratakonda describes a scene change detection algorithm of Lagendijk that selects scene change locations based on minimization of an action integral.
U.S. Pat. No. 6,014,183 of Hoang describes a method and system for retrieving frames of video data from a display control device and automatically detecting scene changes. Hoang detects scene changes by comparing pixel color values in successive frames.
Microsoft MovieMaker™ and Pinnacle Studio DV™ include scene change detection features.
Prior art scene change detection algorithms suffer from a shortcoming in that intermittent effects within a video shot, such as lighting changes, camera flashes, or a bird flying in and out of a scene, can produce significant changes in color histograms and be interpreted falsely as scene changes. Hoang (col. (col. 8, lines 1–16, and FIG. 8) describes examining a predetermined number of frames in order to determine whether a potential scene change should be considered a bona fide scene change. This serves to filter out isolated events such as flashbulbs going off. Hoang compares frames following a potential scene change with an initial frame, immediately prior to the scene change.
Hoang's method works well to filter out abrupt scene changes, such as the change indicated in FIG. 8 of Hoang. However, Hoang's method can fail to detect bona fide scene changes that are gradual. For gradual scene changes, the differences between frames may not be large enough to exceed Hoang's threshold, and, as a result, Hoang may overlook the gradual scene changes.